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The purpose of this study was to determine factors that influenced student 
satisfaction with a new learning management system and to identify which of these 
factors were most important. The data was collected using an an online survey tool 
that was administered to students enrolled in courses designed and taught by 
faculty who participated in a pilot group testing a new learning management 
system (LMS). The findings support previous research in this area, indicating that 
usability of the LMS and availability of technical assistance is strongly correlated 
with student satisfaction. Study findings will be used to improve LMS training and 
institution-wide technology support. 
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learning, course design. 

As universities grow in population size, more emphasis is placed on e-learning. In 
fact, Olgren (2000) indicates that distance learning is growing at an annual rate of 
25% in the United States in both the corporate and academic worlds. Urdan and 
Weggen (2000) define e-learning as the delivery of course content through 
electronic media (e.g. internet, interactive TV, satellite, audio/video tape, etc.). 
Khan (2001) provides multiple synonyms for e-learning, including web-based 
learning, internet-based training, advanced distributed learning, web-based 
instruction, online learning, and open/flexible learning. There are multiple 
definitions and synonyms for distance learning, however, there is a common theme. 
E-learning is a method of course delivery that requires internet connectivity, used to 
deliver content both synchronously and asynchronously via electronic media or 
software to the learner (Allen & Seman, 2009; Ko & Rossen, 2010). In the context of 
this study, web-based learning is defined as instruction delivered to the learner over 
the internet asynchronously, synchronously, or both (Koohang & du Plessis, 2004). 

Much web-based learning is facilitated via a learning management system (e.g.. 
Blackboard, Moodle, and Sakai). One of the most important functions of an LMS is 
its ability to address the needs of the end user, or the learner (Lewis & MacEntee, 
2005). In web-based learning, students must have high self-regulation, motivation 
and commitment to the learning process (Selim, 2007; Artino, 2008). Unfortunately, 
the majority of students do not have those necessary cognitive skills. 
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Therefore, the facilitation of student learning becomes a vital component of online 
teaching. To meet the educational needs of students, it is necessary to ensure that 
e-learning, or web-based learning, is quality learning. In order to keep students 
motivated and committed, student satisfaction is very important and has been 
found to be a major contributor to continued student participation in web-based 
learning (Palmer & Holt, 2009). Furthermore, learner satisfaction has been found to 
be positively correlated with the quality of learning outcomes (Artino, 2008; 
Bolliger & Martindale, 2004; Inan & Yukselturk, 2004; Palmer & Holt, 2009). 

FACTORS LINKED TO LEARNER SATISFACTION 

According to Palmer and Holt (2009) learner satisfaction "with an online 
environment is complex and multi-dimensional and includes a wide range of 
factors" (p. 101). Even so, there are several factors that are common throughout 
the literature. In previous research, these factors have been categorized in various 
ways. Sun et al. (2008) categorized these factors into five dimensions: the learner 
dimension, the instructor dimension, the technology dimension, the course 
dimension, and the design dimension. Menchaca and Bekele (2008) categorized 
these factors into five dimensions as well, including technology-related factors, user 
characteristics, course-related factors, learning approach, and support services. For 
this study, factors were organized into two main categories: learner-related factors 
and learning environment-related factors (Inan & Yukselturk, 2004). 

Learner-Related Factors 

Student satisfaction with web-based learning has been shown to be strongly related 
to prior experience. Selim (2007) found that a student's prior experience with using 
a learning management system is very important to online learning success. After 
surveying students and faculty looking at technical proficiency, Menchaca and 
Bekele (2008) found that at least a basic or moderate technical skill was also 
necessary for success in an online program. A student's participation and utilization 
of the learning management system strongly influences his or her level of 
satisfaction with an online course (Inan, Yildirim, & Kiraz, 2004). Palmer and Holt 
(2009) found that the more students participate frequently online, the more 
satisfied they feel with online courses. Conversely, a student's rare use of the LMS is 
related to feelings of dissatisfaction with online learning (Hong & Koh, 2000; Sun et 
al, 2008; Palmer & Holt, 2009). Basically, the amount of time spent online using the 
learning management system influences student satisfaction. 

Learning Environment-Related Factors 

Technology support is essential to the successful use of the LMS in delivering online 
courses and is strongly related to student satisfaction. Technical assistance has to 
be in place, or the online learning experience will not be successful (Koohang & du 
Plessis, 2004; Selim, 2007). Palmer and Holt (2009) state that access to campus- 
based resources and technical support contributes to learner satisfaction. 
Conversely, Selwyn et al. (2000) found that a lack of technical support contributes 
to learner dissatisfaction and is a significant problem for those participating in 
online learning. One of the major subjects regarding student satisfaction discussed 
in the literature is usability (Nokelainen, 2006; Reeves et al., 2002; Storey, Phillips, 
Maczewski, & Wang, 2002). Usability is defined by Koohang and du Plassis (2004) 
as "the influential dynamic of a product or system's capacity to fulfill the users' 
interactive needs and expectations" (p. 40). 
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Usability is critical to the web-based learning environment and includes attributes 
such as reliability, operability, understandability, functionality, learnability, and 
efficiency (Koohang and du Plassis, 2004). Chiu et al. (2005) suggest that usability 
of the learning management system is a major determinant of student satisfaction 
in online courses. 

Previous studies show that the degree to which a course is online can influence 
student satisfaction (Bolliger & Martindale, 2004; Inan & Yukselturk, 2004). There 
are various degrees of online instruction ranging from hybrid, or a mixture of face- 
to-face and online to distance, or fully online courses. In a qualitative study by 
Zhang and Perris (2004), some students indicated that they were more at ease 
when learning in a face-to-face environment rather than an online environment. 

Thomas (2002) found that 51% of students surveyed preferred face-to-face 
discussions over online discussions. Despite these findings, several studies have 
also noted that the ease and flexibility of fully online courses positively impact 
student satisfaction with online instruction. 

PURPOSE OF THE STUDY 

Learner satisfaction is a key factor in continued participation in web-based learning. 
Therefore it is important to evaluate learner satisfaction with the LMS currently 
being used. This study focused on evaluating learner satisfaction and the factors 
that may contribute to or take away from the learning experience. Specifically, this 
study explored the impact of learner-related (prior experience, frequency of use) 
and learning-environment related factors (delivery format, technical assistance, 
usability) on student satisfaction with a new learning management system and 
identifies which factors were most important. 

METHODOLOGY 

Participants 

This study took place in a southwestern research institution. In the fall of 2008, the 
university began migrating from WebCT 4.1 to Blackboard CE 6. The study survey 
was sent to 2,922 students who participated in a course designed by a faculty 
member in the spring 2009 pilot cohort. Of the 2,922 students, 390 responded, 
resulting in a 13% response rate. Students came from all different disciplines. Of 
the 390 students who replied, 13% were freshmen, 13% were sophomores, 24% 
were juniors, 25% were seniors, and 24% were graduate students. More males 
than females responded; 38% of respondents were female while 62% of the 
respondents were male. 

Data Collection Instrument 

Data for this study was collected through a survey developed by a team of 
professionals who offer teaching assistance and professional development for 
online teachers in the university. Utilizing the online survey tool, Qualtrics, students 
who were enrolled in online and hybrid classes for the Blackboard pilot groups were 
given a link to the survey on the homepage of their course. They were asked to fill 
out the survey via a course announcement and were then redirected to the data 
collection website. The survey had 19 questions. 
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These questions ranged from "How much, approximately, do you access your course 
in a week?" to "Have you previously used a course management system 
(Blackboard, WebCT, Moodle, etc.) in a university course?" Students were asked to 
give basic information such as gender and classification. They were also asked to 
respond to statements using a five-point likert-type scale. For this study, specific 
questions were used, including "When I had problems with the course management 
system (Blackboard), it was easy for me to obtain prompt and helpful assistance" 
and "How easy did you find Blackboard to use?" Students were also asked to rate 
overall satisfaction with the use of blackboard in their course. 

DATA ANALYSIS AND RESULTS 

Ordinary least squares multiple regression was used to determine factors 
influencing students' satisfaction with a new learning management system and 
which factors were most important. The dependent variable, student satisfaction, 
was measured by students' perceptions and satisfaction with the new learning 
management system. 

Among the independent variables were students' prior online course experience, 
frequency of learning management system usage, availability and amount of 
technical assistance, format of the course delivery, and the usability of the learning 
management system. 

The sample size for analyses was 267 representing all students who had completed 
a report on all of the variables used in the analyses. The means, standard 
deviations, and correlations among all the variables are given in Table 1. 


Table: 1 

Correlations, Means, and Standard Deviations (N = 267) 


Variables 

1 

2 

3 

4 

5 

6 

1. Prior Experience 

1 






2. Frequency of Use 

.001 

1 





3. Delivery Format 

-.190 

-.243 

1 




4. Technical Assistance 

.083 

.035 

-.070 

1 



5. Usability 

.077 

.022 

-.062 

.503 

1 


6. Student Satisfaction 

.104 

.094 

-.066 

.559 

.783 

1 

Mean 

1.16 

3.45 

2.58 

3.16 

3.63 

3.57 

SD 

.551 

1.137 

1.212 

1.045 

1.080 

1.099 


The five independent variables were entered into the regression equation 
simultaneously. Preliminary examination of the results indicated there was no 
extreme multicollinearity in the data (all variance inflation factors were less than 3) 
nor were there any influential data points. The regression results indicate that the 
set of independent variables explained 64.3% (/(5,258) = 95.6, p < .001) of the 
variance in student satisfaction; with three of the five variables having a significant 
unique influence on satisfaction (see Table: 2). 
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In order of importance, they are usability {Beta = .67), technical assistance {Beta = 
.22), and frequency of LMS use (Beta= .08). The findings suggest that the usability 
of learning management system and the availability of technical assistance are very 
important factors influencing student satisfaction with online learning management 
systems. 

Table: 2 

Results of Regression of Student Satisfaction 


Variables 

b 

£ 

t 

1. Prior Experience 

.075 

.038 

1.017 

2. Frequency of Use 

.073 

.076 

2.023* 

3. Delivery Format 

.015 

.017 

.437 

4. Technical Assistance 

.228 

.217 

5.142** 

5. Usability 

.683 

.671 

15.942** 

R Square= .643 


*p < .05; ** p < .01 


DISCUSSION 

This study examined factors -usability, frequency of use, delivery format, technical 
assistance, and prior experience- contribute to student satisfaction. Three of the 
five factors examined were found to be important, two of which were strongly 
significant. Usability was found to be very important in contributing to student 
satisfaction. 

The more students view the LMS as usable, the higher their satisfaction with the 
system. This supports the findings of Koohang and Duplassis (2004) and Wu, 
Tennyson, and Hsia (2010). Therefore, it is imperative that an LMS be effective, 
reliable, operable, understandable, functional, learnable, memorable, and efficient. 
Students must feel comfortable with the system in order to focus on learning the 
content rather than system navigation and usability. If it takes more time to learn 
to use the system, or if it is difficult in some way or another, students are less likely 
to be satisfied with the LMS, which in turn would keep them from being satisfied 
with their learning experience. As usability was found to be a significant contributor 
to student satisfaction, it is of utmost importance for an educational institution to 
consider the usability of online learning environments in order to ensure quality 
online learning. 

Technical assistance was found to significantly contribute to student satisfaction. 
Noted by Palmer and Holt (2009), Selwyn (2000), Selim (2007), and others, the 
availability and quality of technical assistance are significant contributors to 
student satisfaction. 

The support students receive from institutional support staff for an LMS is 
invaluable. When using an LMS in an institution, it is important that technical 
assistance be one of the top priorities to ensure student success in an e-learning 
environment. Additionally, student usage of the LMS was found to be another 
important factor impacting student satisfaction. 


193 






The more a student uses the system, the higher his or her level of satisfaction 
(Hoskins & Van Hooff, 2005; Hrastinski, 2008). Students who access their courses 
more may have become more accustomed to the course and may have learned more 
about the use of the LMS. 

Therefore, course designers and instructors should design and deliver instruction in 
ways that encourage students to utilize the LMS more frequently so as to improve 
student feelings of self-efficacy and comfort with the LMS (Inan, Yildirim, & Kiraz, 
2004). 

Students' prior online experience was not found to be important for satisfaction 
with the LMS. However, current college students may be overall, more comfortable 
with computer technology and adapt more easily to an online environment 
(Birdsong, 2009; Koroghlanian & Brinkerhoff, 2008) 

Therefore, prior experience may not have been found to be a contributor to student 
satisfaction as online environments may no longer be such a challenge for students 
to learn. Study findings also did not show any significant relationship between the 
delivery format and student satisfaction. This substantiates the idea that a well- 
designed online course can compete with a face-to-face course. For example, the 
use of communication tools can replace discussion in a face-to-face class. Also, 
students who enroll in online courses usually do so in order to have more flexibility 
in their learning process. So, student satisfaction may not necessarily be tied to a 
face-to-face element in the delivery of the course. 

CONCLUSION 

This study did not attempt to measure all contributors to student satisfaction. 
However, from its findings, student satisfaction can be clearly linked to usability, 
availability and quality of technical assistance, and frequency of use. This research 
has implications for educational institutions wishing to provide students with a 
quality e-learning experience. It is important that the system be usable, that 
students have access to technical support, and that they be encouraged to use the 
system frequently. In this way, quality online learning can be a reality for students 
and for educators. 
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